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ABSTRACT 



In this paper, a mathematical programming approach is 
presented for the assembly of ability tests measuring multiple traits. The 
values of the variance functions of the estimators of the traits are 
minimized, while test specifications are met. The approach is based on 
Lagrangian relaxation techniques and provides good results for the two 
dimensional case in a small amount of time. Empirical examples of a test 
assembly problem from a two dimensional mathematics item pool illustrate the 
method. In the area of ability measurement, Item Response Theory (IRT) is 
usually used as a psychometric theory underlying the test assembly process. 

In this process, three steps can be distinguished. First, an IRT model has to 
be chosen and the items in the item bank have to be calibrated. From this 
item bank, many different tests can be assembled. Therefore, the second step 
consists of specifying the properties of the desired test. One could specify, 
for example, the test length, the desired amount of information, or the 
administration time of the test. The third step of the test assembly process 
is to formulate a model that selects items from the item bank so that test 
specifications are met. A mathematical programming approach is often used for 
this step. (Contains five tables, two figures, and nine references.) 
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Abstract 

In this paper a mathematical programming approach is presented for the assembly of 
ability tests measuring multiple traits. The values of the variance functions of the estimators of 
the traits are minimized, while test specifications are met. The approach is based on Lagrangian 
relaxation techniques and provides good results for the two dimensional case in a small amount 
of time. Empirical examples of a test assembly problem from a two dimensional mathematics 
item pool illustrate the method. Keywords: Lagrangian relaxation, mathematical programming, 
multidimensional IRT, test assembly, test design. In the area of ability measurement Item 
Response Theory (IRT) is generally used as a psychometric theory underlying the test assembly 
process. In this process three steps can be distinguished. First an IRT-model has to be chosen 
and the items in the itembank have to be calibrated. From this itembank many different tests can 
be assembled. Therefore, the second step consists of specifying the properties of the desired 
test. One could specify for example the testlength, the desired amount of information or the 
administration time of the test. The third step of the test assembly process is to formulate a model 
that selects items from the itembank so that the test specifications are met. A mathematical 
programming approach is often used for this step. 
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Multidimensional Test Assembly based on Lagrangian Relaxation Techniques 



In the area of ability measurement Item Response Theory (IKT) is generally used as 
a psychometric theory underlying the test assembly process. In this process three steps can be 
distinguished. First an IKT-model has to be chosen and the items in the itembank have to be 
calibrated. Form this itembank many different tests can be assembled. Therefore, the second 
step consists of specifying the properties of the desired test. One could specify for example the 
testlength, the desired amount of information or the administration time of the test. The third 
step of the test assembly process is to formulate a model that selects items from the itembank 
so that the test specifications are met. A mathematical programming approach is often used for 
this step. 

The idea of using a mathematical programming approach was first suggested by Yen 
(1983). Ever since, a number of researchers have shown various LP algorithms and heuristics 
to solve test assembly problems. Still much research is done in this field of interest. Recently 
van der Linden (1996) published a paper on the subject of assembling tests measuring multiple 
traits. Although an algorithm to solve the assembling problem was provided, manual correction 
was needed to find the optimal solution. Therefore, the purpose of this study was to find a 
heuristic to solve the problem of assembling tests measuring multiple traits. An approach based 
on Lagrangian relaxation was used. 

A Linear Logistic Multidimensional IRT Model 



The IKT model 

The model considered in this paper is a generalization of the two-parameter logistic 
model Lord (1980) to the multidimensional case and can be formulated in the following manner: 

PiiBj) s P(Uij = 1 | (a,, di, Oj)) (1) 

ei&iBj+di) 

= 1 +^^- 4 -*)’ 

where Uij represents the response of person j = 1 ... N to item i = 1 ... n, is the vector 
of discrimination parameters of item i along the abilities 6j \ . . . 0 jm , m is the dimensionality 
of the ability space, and is the parameter representing the difficulty of the item. P(Uij = 
l|(ai, di , 0j)} is the probability of a correct response (score of 1) for person j on test item i, and 
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a* • Oj is the inner product of the vector of discrimination parameters of item i and the ability 
vector 9j for person j. The item parameters are supposed to be known and the model is used 
to estimate the ability vectors 9j from a realization of the response variables Uij ~ Uij for 
i = 1 .. .n and j = 1 ... N. 

This multidimensional IRT model was also described by Reckase (1997). In this 
article several assumptions underlying this model were presented. First of all, the monotonicity 
assumption implies that, with an increase in the ability of the examinee, the probability of 
obtaining a correct response to a test item is non-decreasing. Second, the probability of 
obtaining a vector of responses is equal to the product of the probabilities of obtaining these 
responses. This assumption is called the local independence assumption. The third and last 
assumption deals with the derivatives of the function describing the model. These derivatives 
have to be properly defined. 



Variance functions 



Fisher’s information matrix is defined as: 



’ d 2 In L 
d$\ 



1(9) s -E 



d 2 In L 
dO^dOi 



d 2 In L 
dOxdQn 



a 2 In L 

~9%r 



were the likelihood equation L for the model in Equation 1 is given by: 



( 2 ) 



L = P[P( Uij |(a i) d < ,0)). (3) 

i=l 

According to van der Linden (1996) and using the notation Pi=Pi(9) and Q { = 1 — P iy Fisher’s 
information matrix for this model can be formulated as: 



m = 



‘Y.U tiiPiQi ••• E7=t auOmiPiQi ■ 
. Er=i a. U a m iPiQi . . . 5Z”=1 a mi P iQi 



(4) 



Because Fisher’s information function is a matrix instead of a scalar in the 
multiparameter case, the (asymptotic) variances of the MLEs of the ability parameters 0 m 
are not given by the reciprocals of the diagonal elements of the information matrix, but by 
the diagonal elements of the variance-covariance matrix. This matrix is the inverse of Fisher’s 
information matrix. For notational simplicity we consider the case that m — 2. In this case the 
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inverse matrix is of the following form: 



V(9\ 9) = 



\m\ 



e?=i <4iPiQi 

~YX=i a ua2iPiQi 



- E"=i «“2 «p«<3« 
E”=i AAQi 



(5) 



where|/(0)| is the determinant of the matrix in Equation 4. It should be noted that the 
variance of 0* not only depends on 6\ but also on the values of 02 - The variance functions 
Var(0 k \0) = EL: “iPiQi/ U(9)\ simplify to: 



Varfay 9) = (J2 a l P >Qi) 

t= 1 

and 



^ a u p iQi 



<i=l 




n -i 



E a-ufyiPiQi 



*t=l 



( 6 ) 



Kar(0 2 |0) = (^a? i P i Q i ) 
1=1 






,t=l 



,i=l 



'EauavPiQi 



V t=l 



-1 



a) 

The objective of test assembly problems is to minimize the values of these variance functions. 
Therefore a multidimensional maximin model was formulated. 



A Multidimensional Maximin Model 



Introducing a decision variable x t , z = 1 , where = 1 if item i is in the test, 
X{ = 0 if item i is not in the test and / is the number of items in the itembank. Equation 6 and 
Equation 7 can be rewritten such that: 



Var(6i\0) = 



l -l 



^ ] &2j P iQi^i ( ^ ^ a ’li a ’2i P iQj3'i j / I 0>n P iQi^i 



( 8 ) 



t=i 



. 1=1 



<i=l 



and 



Var{0 2 \9) = 



T -1 



^ ^ Qli^igi^i ( ^ ] fl.iiQ.2i-P jQjXj | / ( U'2t^Qi a 'i 



(9) 



t=i 



,i=i 



,t=i 



Objective function 

The objective of the test assembling problem is to minimize the values of both variance 
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functions. This problem is in the class of multi objective decision problems. Problems in this 
class can be solved in several ways, see also Hwang and Masud (1979). In this article an 
approach based on combining both functions into one objective function was chosen. To find 
a good combination of both variance functions the effect of the sums on the variance functions 
was investigated. About the variance functions the following observations can be made. From 
Equation 8 it can be derived that the value of Var(9\ |0) for a given 6 decreases when 

/ 

( 10 ) 

1=1 



increases in value. 



decreases in value and 



&lt 0*2^ iQi^i 



v i=l 



53 a U P iQi X i 
1=1 



(ID 



(12) 



increases in value. 

It should also be noticed that the value of Var(9 2 \0) for a given 6 decreases when 



i 

^ ^ iQ{Xi 
1=1 

increases in value, 



^ iQiXi 



<t=l 



(13) 



(14) 



decreases in value and 



/ 

53 0 -ltPiQiXi (15) 

1=1 

increases in value. So, both variance functions show the same behaviour for these sums. 
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Therefore, it seems reasonable to choose the function 






£<=1 O-liPiQiXi + E[=l aliPiQiXi ’ 



which shows the same behaviour, as an objective function. 



(16) 



The model 

The objective function defined in Equation 16 is a continuous function of the variables 
(0i>0 2 )- However, in the test assembling process it suffices to optimize this objective function 
for a grid of points. Therefore, the following mathematical programming problem has been 
chosen, which minimizes the maximum value of a combination of the variance functions for a 
set of 0-points (0i p , 0 2q )> P = 1, • • • , P and q = 1 , . . . , Q: 



min max 



(]C*=1 a li a 2iPi(Q\s) 02t)Qi{0\s, 02t)Zi) 



i moA >■ \ / v j 

tzh T [ZLi<Pi(^M( 0 u,O 2 t)xi) +i (^ l <&Pi(9u,9*)Qi(0u,0v)*i) 

subject to 



(17) 



i 

^ ^ Xt = Tly 
t=l 



(18) 



Xi€{ 0,1}, < = I, (19) 

where c in Equation 17 is a weighting factor. By varying this weighting factor one can influence 
the relative importance of the two sums in the denominator. Equation 18 specifies the testlength, 
but, of course, other constraints can be added to this problem. Unfortunately, the objective 
function, Equation 17, is a complicated function of the variables x { . By creating three new 
variables y , K\ and K 2 the problem can be restated as: 



min 



K x +K 2 



( 20 ) 
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subject to 



/ 

0*)Qi(0u, @ 2 t)%i — y < 0, s = 1 . . . 5, t = 1 . . . T, (21) 

1=1 

/ 

Y, a l P ^MQ^Mxi>K u s = l...S,t= (22) 

i=l 

/ 

' s ^al i Pi{ 6 Xs , 6 2 t)Qi{ 0 u, 62 t)xi > cK 2 , s = l...S,t= 1...T, (23) 

i=l 

1 

^ ~2,Xx=n , (24) 

1=1 

Xj E {0,1}, i = 1 . . . n, (25) 

y,*i,* 2 >0. (26) 



The constraints. Equations 21, 22 and 23, must be satisfied as equalities at optimality. 



* Other models 

The following multidimensional maximin model to solve this problem, was described 
in van der Linden (1996): 



min y 



(27) 



subject to 



/ 

Y^auaKPiiOu'dMie^e^Xi-y < 0, s = 1 . . .S,t = 1 . . .T, (28) 

1=1 
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i 




i 




t=i 



i 






(31) 



Xi € {0, 1} , t = 1 . . . /, 



(32) 



y >0. 



(33) 



The basic idea is to run this model, while the variables K\ and K 2 are systematically varied, until 
the optimal solution is found. The main difference between this model and the model described 
by Equations 20 until 26 is that the objective function. Equation 20, is linearized and that this 
problem has to be solved many times for many combinations of (K X) K 2 ). Furthermore, the 
factor c is not necessary, because the ratio of K\ and K 2 can be used as a weighting factor. 

Several more models can be formulated that solve the same problem. However, the 
purpose of this study is to find a heuristic which provides good results in a small amount of 
time. Therefore it might be better not to eliminate the variables and to look for a different way 
to handle this problem. In this study an approach based on Lagrangian Relaxation was chosen. 



The problem stated above is in the class of non-linear mixed integer programming 
problems. The problem is NP-hard, which means that it can not be solved in polynomial time. 
In Armstrong, Jones and Wang (1995) an approach, based on Lagrangian Relaxation techniques, 
is described to solve various test assembly problems. 

Lagrangian Relaxation techniques are based on the idea that a good lower or upper 



Application of Lagrangian Relaxation to this Problem 
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bound to the solution of a mathematical programming problem can be obtained by relaxing 
some constraints. These constraints are not just thrown away, but are brought into the objective 
function by a correction term. This correction term is a linear combination of the constraints. 
The remaining relaxed problem is solved several times using various linear combinations of the 
constraints to obtain a good approximate solution to the original problem. The main difficulty 
in this technique is the calculation of the correction term. 



Notational simplification 

Before we try to use this approach for our problem we introduce the following 
definitions for notational simplicity: 



Q-i 1 &2\P \j(@ llj > @21j)Qlj{@llj , @21j) 



0'ln0'2nPnj{@llji @21 j)Qnj{@ ilj j @21 j) 



Aij 



— a U a 2 \Pij(@iSj, @ 2 Tj)Qlj (@\Sj) @2 Tj) 

a nPij(@nj,@2ij)Qij(@nj>@2ij) 

a \lPlj(@lSj,@2Tj)Qlj(@lSj,@2Tj) 

a 2\Plj{@l\j,@2\i)Q\j(@\\j,@2\j) 



— Q'\na2nP rij{@\Sj) @2Tj)Qnj(@lSj,@2Tj) 

®lnPnj i@llj) @21 j)Qnj(@ 11 j) @21 j) 

a 1 n Pn j (@ 1 S j > @2T j ) Qn j ( @1 Sj > @2 Tj ) 
a 2 nPnj (@11 j > @21 j)Qnj (@llj > @21 j) 



a 21 P lj{@lSj , @2Tj)Qlj {@lSj , @2Tj) 



’ a 2nPnj(@lSj,@2Tj)Qnj(@lSj, @2 Tj) 

(34) 



were j = 1, .., 3 ST and i = 1, n, and 

.* • * i -y> #i, * > Ki,cK 2 , • • • , c/T 2 ] . (35) 



So, the constraint Ax > b yy K u K 2 ls equivalent to the constraints defined in Equations 28 until 
30. Using this notation, the maximin model simplifies to: 



min 



K\ + K 2 ' 



(36) 



subject to 



Ax > b y tKuK 2, 



(37) 



^2 Xi = n > (38) 

»=i 
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e {o, 1} , i = l ... 7i, 



(39) 



KuK 2t y > 0 . 



(40) 



Lagrangian Relaxation 

A Lagrangian Relaxation of this problem can be obtained by relaxing Equation 37, 
i.e. by relaxing the constraints due to a rewriting of the objective function. Other constraints, 
e.g. the constraint specifying the test length, may not be relaxed. Otherwise, the test would no 
longer meet its specifications. The resulting Lagrangian Relaxation problem can be stated as: 



The elements of u are called Lagrangian multipliers. 

Some remarks can be made about this mathematical programming problem. Looking 
closer at Equation 41, it can be observed that the objective function can be split in two 
independent parts. Consider Equation 41: 




(41) 



subject to 



/ 




(42) 



*=i 



Xi € {0,1} , i = l.../, 



(43) 



u j} Ki,K 2 ,y > 0, ; = 3 ST. 



(44) 



mi" k^Tk 2 + uT ( b “ Ax ) 



( 45 ) 
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where the values of vector b depends on y , K x and K 2 . Equation 41 can be rewritten: 



min — — + u T b - u T Ax 

K x + X 2 



(46) 



+ u T bj — (u t Ax) (47) 

min / u (y, K x , K 2 ) - g u (x) (48) 

where y , K X ,K 2 > 0, and the conditions defined by Equation 42 and Equation 43 have to be 
met. So, the functions / u and g u can be minimized separately. The result of this separation 
is that the problem is split in a continuous non-linear optimization part and a zero-one linear 
programming part. The remaining problem is to find a good u vector. 

Subgradient optimization 

A general approach to the problem of finding a good u-vector is subgradient 
optimization. This technique is based on two observations concerning the Lagrangian relaxation 
of a mathematical programming problem. If the following problem has to be solved: 

z P — min c T x (49) 

subject to 

Dx < e, (50) 

Fx < g, 

where Dx < e defines a set of constraints and Fx < g defines a set of constraints, a Lagrangian 
relaxation of this problem is: 

z D ( u) = min{c T x + u T (Dx — e)} (51) 






+ k 2 
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subject to 



< g, 



(52) 



If the equation Fx < g defines a finite set of points the Lagrangian relaxation can be written 
as: 



and the following observations can be made. Notice that zz?(u) minimizes a finite number of 
linear, therefore concave functions of u. Because of this, there exists a subgradient of z D ( u) for 
every u, see also Definition 1 . Further, there exists at least one u* which maximizes z D (u) . For 
a more detailed description of these observations see also Dirickx, Baas and Dorhout ( 1987). 

Definition 1 A vector r 6 FT 1 is a subgradient of a concave function /( u) in u G IF*, if for 
every u € FT 1 



Application of subgradient optimization 

Applying this technique to the problem stated in Equation 36 until Equation 40, 

equation 



z D { u) = min {c T x t + u T (Dx t - e)} 



(53) 



/(u)</(u)+r>-u). 



Ax > b y,Kl,K2 



(54) 



can be seen as Dx < e and the equations 



i 




(55) 



Xi 6 {0, 1} , i = 1...I 



(56) 
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u h K u K 2i y>^ j = 1...3ST 



(57) 



can be seen as Fx < g. 

To check if the subgradient optimization technique can be applied to the problem 
described in Equations 41 until 44 we have to ensure that all conditions are met. First, the 
objective function of the relaxation has to be a linear function of the variable u. From Equation 
41, it can be derived that the vector u is only involved in an inner product, so this condition is 
met. Second, the feasible region has to be a finite set of points. The feasible region is defined 
by Equations 42 until 44. Unfortunately, K\, K 2 and y are continuous variables. However, as a 
result of splitting the optimization problem, the optimal values of y , A^and K 2 can be computed 
separately and these variables can be fixed at their optimal values, so the feasible region is a 
finite set of points. Now all conditions are met and a description of the procedure can be given. 
The following procedure outlines the search for a good u-vector: 

• Step 0. Set k = 0, let z be an upper bound to zd( u), and create an initial vector uo- 

• Step 1. Calculate the subgradient r* by solving the mathematical programming problem 



If y, K\ % K 2 , x is the optimal solution to this problem, define r* = Ax k — h(y,K\,K 2 ). 

• Step 2. t k = where 0 < n k < 2. 

• Step 3. u;t +1 = u* + t k ■ r*. 

• Step 4. k := /c + 1. 

• Step 5. If k < 75 go to step 1, else go to step 6. 

• Step 6. End of the u-search, set u = u 75 . 

The reason for terminating this procedure when k > 75 is that the values of the 
elements of the correction vector u strongly depend on the parameter fi k , which, in turn, depends 
on the stopping criterium k = 75. The larger the stopping criterium the more elements of u equal 
to zero and the higher the values of the remaining elements. In general, the value of the stopping 
criterium should be in the interval [40.. 125], and A: = 75 seems a good choice. For a detailed 
description and validation of this subgradient procedure see Dirickx, Baas and Dorhout (1987) 
and Held, Wolfe and Crowder (1974). 



min 




(58) 
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Empirical Examples 

Two examples are presented. First, the LR-problem stated in Equation 41 till Equation 
44 was solved for data from an ACT Assessment Program Mathematics Item Pool. Second, the 
same problem was solved for the same data, but with an additional set of constraints. 

The item pool consisted of 176 items, to which an acceptable fit was shown by a two 
dimensional version of the model in Equation 1. The items were classified according to content 
specification and skill. 

The above described method was used to solve the models on a PC with a Pentium 
processor (130 mHz). Heuristic seven of the program Contest was used to solve the zero-one 
linear programming parts of the second example. 



Example 1 

It was attempted to assemble a test for both ability estimators over the complete grid 
of points defined by 0i,0 2 = — 1,0, 1. The test should contain 25 items. The upper bound to 
the Lagrangian relaxation z D {\x) was set equal to z = 3 and the weighting factor fi k in Step 
2. of the above described algorithm was set equal to 0.2 and halved every ( sto ai in £ ^ tCTlum ) ^ 7 
iterations. The values of the upper bound and the weighting factor depend on the problem and 
should be found empirically. The resulting variance functions are shown in Figure 1. 

To test the robustness of the above described method attempts were made to assemble 
tests for several initial vectors u 0 . The mean values (/z* , /z 2 ) and the standard deviations (<Ti , < 72 ) 
of the values of both variance functions over nine points of the grid were used as a measure to 
compare the results. Table 1 shows the results and the CPU-time for several starting values uo. 

Table 1 



Results for starting values up and k = 75 



Uo 


Mi 


M2 


CT\ 


<72 


M 


<7 


(l + <7 


time ( 5 ) 


0.05 


1.081 


0.889 


0.153 


0.188 


0.985 


0.341 


1.326 


4.73 


0.10 


1.089 


0.858 


0.146 


0.182 


0.974 


0.328 


1.302 


4.89 


0.15 


1.135 


0.876 


0.157 


0.180 


0.989 


0.337 


1.326 


4.94 


0.20 


1.157 


0.888 


0.147 


0.187 


1.023 


0.334 


1.357 


4.89 


0.25 


1.234 


0.846 


0.163 


0.177 


1.040 


0.340 


1.380 


5.05 


0.30 


1.275 


0.845 


0.184 


0.179 


1.060 


0.363 


1.423 


4.94 



The CPU-time needed to find a solution is only five seconds, which shows that this 
method is rather quick. Because of small differences in the resulting fj, 4- <7, the method seems 
rather robust to small modifications of the starting values uq. The method performs best when 
0.05 < u 0 < 0.15. To examine the stopping criterium ( k = 75) the mean values and standard 
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deviations of the values of both variance functions were compared for several values of k. The 
results are shown in Table 2. 

o 

Table 2 



Results for several values of k and starting values Up — 0.15 





V 


a 


M + o 


k = 50 


1.020 


0.365 


1.385 


k = 75 


0.989 


0.337 


1.326 


k = 75 


0.992 


0.375 


1.367 



It can be concluded that the value of fi + a does not change much for k > 50, although 
k = 75 performs best. On the other hand, if attention is paid to the values of the correction 
vector u y it can be seen that many violations of the relaxed constraints are corrected by a small 
term if k = 50. On the other hand, if k = 100, a few violations are corrected by a big term. 
Therefore it seems reasonable to choose k = 75, which is somewhere in between, as a stopping 
criterium. 



Example 2 

Both content and skill constraints were added to the LR-problem described in Equation 
41 till Equation 44: 

(1) The test should contain at least 2 plane geometry, 2 pre-algebra, 2 elementary 
algebra, 2 coordinate geometry, 2 trigonometry, and 2 intermediate algebra items. 

(2) At least 7 basic skill items, 7 application items, and 2 analysis items should be 
included in the test. 

This results in the following additional constraints: 



53 x * 


> 


<€ Vpg 




53 x ' 


> 






53 Xi 


> 


iev EA 




53 Xi 


> 


ievcc 




13 Xi 


> 


Vtg 




53 x < 


> 


<€Vm 





(59) 
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5Z ** 

iZV BA 



5Z x * 



5Z x * 



> 7, 

> 7, 

> 2, 



where for example Vpc is the set indices of the items with content classification plane 
geometry (PG). The same specifications as in example 1 were used. The resulting variance 
functions are shown in Figure 2. To compare the results a description of the results of this 
example is given in Table 3. 

Table 3 



Results for starting values up and k = 75 



Uo 


Mi 


M2 




<?2 


M 


a 


^ 4- a 


time (s) 


0.05 


1.099 


1.283 


0.154 


0.208 


1.191 


0.362 


1.553 


23.12 


0.10 


1.094 


1.415 


0.150 


0.230 


1.254 


0.380 


1.634 


23.56 


0.15 


1.094 


1.415 


0.150 


0.230 


1.254 


0.380 


1.634 


23.57 


0.20 


1.158 


1.337 


0.168 


0.214 


1.248 


' 0.382 


1.630 


23.89 


0.25 


1.158 


1.337 


0.168 


0.214 


1.248 


0.382 


1.630 


22.52 


0.30 


1.158 


1.337 


0.168 


0.214 


1.248 


0.382 


1.630 


22.63 



As can be seen, the values of fi 4- cr increase in comparison to the values of fi + cr in 
example 1. This is because of the additional constraints. Because only small differences in the 
values of [i + a occur, the method is rather robust to a little modification of the initial values uo 
even if there are additional constraints. The CPU-time to find a solution is about twenty three 
seconds. Most of this time is needed to switch to Contest for solving the linear programming 
part of the problem. However, twenty three seconds is still pretty fast for assembling a test 
measuring multiple traits. To examine the stopping criterium (k = 75), the results for several 
stopping criteria are showed in Tkble 4. 

Table 4 



Results for several values of k and starting values uo = 0.15 





M 


a 


fi + cr 


A: = 50 


1.302 


0.438 


1.740 


k = 75 


1.255 


0.373 


1.628 


k= 100 


1.256 


0.379 


1.635 



As can be seen, for this example the best results were obtained for k = 75. And 
choosing A; = 75 as an stopping criterium seems a good choice. 



Multidimensional Test Assembly - 18 
In both examples no attention was paid to the differences in the shapes of the resulting 
variance functions. However, if it is important that both resulting variance functions are of 
the same shape one could use the parameter c in Equation 23 as a stabilizing factor. For the 
problem stated in Example 1, the results for various values of the factor c are shown in Table 
5. The shapes of both variance functions are represented by the values of + <j\ and /z 2 + ^ 2 - 
The value of + au decreases when the value of c decreases and the value of /i 2 + cr 2 increases 

when the value of c decreases. The difference between both shapes will be minimal for c « 0.98. 
However, it has to be emphasized that the optimal value of c is conditional on both the problem 
and the itembank. 

Table 5 



Effect of the stabilizing value c 



c 




^2 


V\ 


&2 


Ml +<Tl 


1^2 + a 2 


1.25 


1.623 


0.742 


0.227 


0.167 


1.850 


0.909 


1.10 


1.218 


0.799 


0.160 


0.196 


1.378 


0.995 


1.05 


1.135 


0.876 


0.156 


0.220 


1.291 


1.096 


1.00 


1.135 


0.876 


0.156 


0.220 


1.291 


1.096 


0.95 


0.977 


1.045 


0.129 


0.276 


1.106 


1.321 


0.90 


0.918 


1.181 


0.116 


0.337 


1.034 


1.518 


0.75 


0.918 


1.181 


0.116 


0.337 


1.034 


1.518 



Conclusion 



Although it seems quite obvious to use the variance functions as a base for assembling 
tests measuring multiple traits, the remaining mathematical programming problems is hard to 
solve. In this paper a method is presented which offers the possibility of computing a good 
upper bound to this problem. Especially the opportunity of splitting the nonlinear programming 
problem in two problems which are easy to solve, simplifies the calculation process. 

Therefore, it can be concluded that an approach based on Lagrangian relaxation is a 
good method for solving the problem of assembling tests measuring multiple traits. On the other 
hand a rather general approach to find a good u vector was used and a more specific algorithm 
might further reduce the CPU-time and improve the test assembling process. 
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Figure Captions 

Figure 1 . Variance functions of the MLEs of the ability parameters for the tests assembled in 
Example 1 , when the stopping criterium was set equal to k = 50 (at the top), k = 75 
(in the middle) and k = 100 (at the bottom) and the starting values of the correction 
vector were set equal to uo = 0. 1 5. 

Figure 2. Variance functions of the MLEs of the ability parameters for the tests assembled in 
Example 2, when the stopping criterium was set equal to k = 50 (at the top), k=15 
(in the middle) and k- 100 (at the bottom) and the starting values of the correction 
vector were set equal to uq = 0. 1 5. 
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